f 0.^ 

N90-16216 


FROM PRIMAL SKETCHES TO THE RECOVERY OF INTENSITY 
AND REFLECTANCE REPRESENTATIONS 

Rachel Alter- Gartenberg* 

Old Dominion University, Norfolk, Virginia 

Ramkumar Narayanswamy 

Science and Technology Corporation, Hampton, Virginia 

and 

Karen S. Nolker 

Computer Sciences Corporation, Hampton, Virginia 


ABSTRACT 


A local change in intensity (edge) is a characteristic that is preserved when an image 
is filtered through a bandpass filter. Primal sketch representations of images, using the 
bandpass-filtered data, have become a common process since Marr proposed his model for 
early human vision. In this paper, we move beyond the primal sketch extraction to the 
recovery of intensity and reflectance representations using only the bandpass-filtered data. 

Assessing the response of an ideal step edge to the Laplacian of Gaussian (v^^) filter, 
we have found that the resulting filtered data preserves the original change of intensity that 
created the edge in addition to the edge location. Using the filtered data, we can construct 
the primal sketches and recover the original (relative) intensity levels between the boundaries. 
Similarly, we found that the result of filtering an ideal step edge with the Intensity- Dependent 
Spatial Summation (IDS) filter preserves the actual intensity on both sides of the edge, in 
addition to the edge location. The IDS filter also preserves the reflectance ratio at the edge 
location. Therefore, we can recover the intensity levels between the edge boundaries as well 
as the (relative) reflectance representation. The recovery of the reflectance representation is 
of special interest as it erases shadowing degradations and other dependencies on temporal 
illumination. 


This method offers a new approach to low-level vision processing as well as to high data- 
compression coding. High compression can be gained by transmitting only the information 
associated with the edge location (edge primitives) that is necessary for the recovery process. 


1. INTRODUCTION 

Primal sketches have become an important method of image description for low-level 
vision. One approach commonly used to produce these sketches is to bandpass filter the 
image data and then use the antisymmetrical signals created around intensity transitions 
(edges) to find their boundary location. We call the antisymmetrical signal a Mach-band 
pattern because it resembles the visual perception of an edge known as Mach-bands.^ In this 
paper, we show that the Mach-band patterns contain more information about the original 
target than just the edge location. This additional information allows us to move beyond 
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the extraction of primal sketches to the recovery of intensity and reflectance representations. 
Figure 1 demonstrates the recovery process of the intensity representation from the bandpass- 
filtered data for both a computer-generated target and a sampled image (e.g., image that 
is degraded by aliasing, blurring, and noise). The bandpassed images (b) of the targets 
(a) exhibit the familiar Mach-band patterns around the intensity transitions. Using the 
information contained in the Mach-band patterns we can recover the locations of the intensity 
transitions and extract the primal sketches (c) together with the actual change of intensity 
there. The recovery process uses this additional information to recover the original target 
(a), using only the bandpassed data (b) as is illustrated in (d). 

The conditions that allow us to recover a signal from partial information and specifically, 
the relationship between signals and their zero crossings, have been of considerable interest 
in the past. Logan^ has set the conditions under which one-dimensional bandpass signals 
are uniquely specified by their zero crossings. Curtis et al.^ set the conditions under which 
real, continuous, periodic, band-limited two-dimensional signals are specified from the zero- 
crossing locations of the real part of their Fourier transform. They applied their results to 
recover simple images from their threshold crossings. Independently, Yuille and Poggio,"^ 
and Hummel^ showed that in the absence of image-gathering degradations, a target could 
be uniquely recovered from the information contained in its second derivative. 

In this paper we assess the response of an ideal step edge to two models for retinal 
processing in human vision. We have found that the Mach-band pattern that results from 
filtering an ideal step edge with the filter^ preserves the original change of intensity that 

created the edge, in addition to the edge location (the zero-crossing location). Therefore, 
we can construct the primal sketches and recover the (relative) intensity levels between 
the boundaries. Similarly, we have found that the Mach-band pattern that results from 
Altering an ideal step edge with the IDS filter^ preserves the actual intensity on both sides 
of the edge in addition to the edge location (the one crossing location). This Alter also 
preserves the reflectance ratio (Weber fraction) at the edge location. Therefore, we can 
recover the intensity levels between the edge boundaries as well as the (relative) reflectance 
representation. 

Our recovery method is local and uses the information contained in each edge element 
explicitly, recursively and independently. Therefore, this recovery process is quick and 
practical, with no need for extra memory, other than the storage for the recovered image 
itself, nor any extra calculations or processing. 


2. PRIMITIVE EXTRACTION FROM THE LAPLACIAN 
OF GAUSSIAN FILTER 


A. The Filter 

The model of lateral inhibition in early human vision processing and the assumption of 
white stationary Gaussian noise as a model for the natural noise source motivated Marr and 
Hildreth^ to develop the spatially invariant operator 


r{x,y\a) 



( 1 ) 
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where + xp' and a is the standard deviation of a normal distribution (Fig. 2). It 

is convenient to normalize the spatial variables relative to the sampling interval. Thus, for 
(T = 1, the standard deviation of the Gaussian is equal to the sampling interval. 

The Gaussian is the only filter that guarantees a nice scaling behavior of the zero and 
level crossings of the linear differential operators.'^ It also is localized simultaneously and 
optimally in the spatial and spatial-frequency domains. The operator is a linear 

isotropic bandpass filter that inherently satisfies the proper sequence of smoothing and 
differentiating for ill-posed differentiation problems (i.e., differentiating noisy data), and 

• JR 

it assures smooth and stable zero-crossing curves. 


B. Step Edge Response 

Isotropic filters allow a one- dimensional change of intensity from / to to simulate 

an ideal step edge. The response of this edge to the filter, as given by 


S(r{x]a,AI) = 


A/ • X 

rexp 



( 2 ) 


is the familiar Mach-band pattern around the edge boundary that crosses zero exactly at the 
location of the change of the intensity. The corresponding peak and trough are symmetrically 
located &t x = a and x = —a taking the values of 


Sff{±a) = ± 


A/ 

y/2'nea^ 


( 3 ) 


The corresponding change of intensity becomes 


AI = 


y/2j^a\Sg{<T) - 
2 


( 4 ) 


Consequently, we have shown that the bandpassed data preserves the edge location and 
the change of intensity across it for spatial details that are at least 3(7 wide (ideal case). Thus, 
when the bandpassed data Sa{x,y) is the only information available, the change of intensity 
associated with each edge element may become part of the image primitives in the low- 
level processing. The low-level processing consists of (1) detecting the zero-crossing location 
{^oiyo)i (2) estimating the local edge direction 0, (3) measuring the values of Sa-{x,y) at 
the points (xo^t <7sin0, j/o ± acosO), and (4) recovering A/ using Eq. (4). The primitives 
{xoiVo)-, 0-, and A/ are used later to recover the (relative) intensity of the original image. 

For the nonideal case, where a detail is insufficiently coarse relative to the spread of 
the filter, or where two edges cross each other or form a corner, it may often be possible 
to approximate A/. This approximation is achieved by measuring Sa{x,y) at the points 
{xq i r sin 0, j/o i r cos 0) where r < a. Eq. (4) then takes on the form 
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A/ = 


(5a) 


y/^a\Sff{r) - 5cr(-r)] 

2rexp (-^) 

when S(r) and S{—r) are available and are approximately antisymmetrical and 


rexpC-^) 


or 


A/ = 


-rexp (-^) 


when only Sa{r) or Sa{—r) can be reliably measured. 


(5b) 


C. Normalized Response 

The Mach-band patterns at different scales (different a) differ from each other not only 
by their spreads but by their amplitudes as well. The bandpass filter can be normalized 
in such a way that the response of an ideal step edge to the \/‘^G filters at different scales 
will have the same amplitude and will differ only in their spreads. The scale Sff for which 
the amplitude of the Mach-band pattern is exactly A/ [i.e., A/ = S{Sa) — 5'(— ^o-)] can be 
obtained from Eq. (4) as 





( 6 ) 


To maintain constant amplitude A/ , regardless of the operator size (i.e., the choice of 
Sff), the bandpassed image should be multiplied by the normalization factor Figure 3 

shows the response of the different operator sizes to an edge after normalization. The 
amplitude of the Mach-band pattern is exactly A/ for all the responses. The only difference 
between them becomes the distance between the peak and trough and the zero crossings. 

Consequently, for the ideal case, we can extract the change of intensity A/ from the 
normalized bandpassed data, S, by the relationship 


A/ - 5(a) - 5(-a). 


( 7 ) 


Similarly, for the nonideal case, we can extract A/ from the relationship 


A/ = -5(r)exp 
0 



( 8 ) 


where b = a Jr, and r = a refers to the ideal case. 
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The recovery of edge primitives obtained for the continuous and normalized response of 
an ideal edge to the filter is in practice constrained by the sampling interval. This 

constraint is part of the imaging system and the inevitable transformation from continuous 
targets to their corresponding digital representations. It is interesting to observe that the 
scale (To corresponds to the size of the smallest scale with which edges can be reliably detected 
relative to the sampling interval of the image-gathering system. Therefore, (Tq determines 
the resolution of the recovery process. According to Marr et al.,^ the smallest operator has 
a standard deviation a= 0.69 relative to a normalized sampling interval. Furthermore, Huck 
et al.^®~^^ have demonstrated that the resultant trade-off between aliasing and blurring in 
the image gathering for this response maximizes the acquired information density for high 
signal-to-noise ratios. The resolution of the recovery is constrained mostly by the sampling 
interval and not by the Gaussian blur represented by <To. In practice, it is preferable to let a 
be 0.75 instead of 0.69. This slight increase in size appreciably reduces aliasing degradations 
for some signals. 


D. The Recovery Process 

The edge primitives extracted by the low-level processing just described, associated each 
edge point {xo,yo) with its local edge direction 9 and its local change of intensity A/. In 
this section we present a method for recovering the (relative) intensity representation of the 
original image from these primitives. At first, we assign one of the regions of the image 
with an arbitrary initial intensity value Iq. From this region onward, we spread the values 
of 7 A/ and I toward the peak and trough of the Mach-band pattern, namely, in the 
(xq ± r sin 9, j/o ± r cos 9) directions. Spreads from different edge points toward the same 
region are averaged. Consequently, the image is recovered constructively, starting from one 
of the regions of the image. Each edge point joining the process provides a step towards 
the final representation of the recovered image. Theoretically, one estimate per region is 
sufficient to recover its (relative) intensity. However, we use estimates from all the ideal edge 
elements to attenuate tne local error of each estimate. The recovery can be correct only up 
to a shift constant that is a function of the difference between the initial value Iq and the 
true intensity that corresponds to this starting region. The recovery process is quick and 
practical. It does not need high-order polynomial representations to describe the image,^’'^ 
nor does it need any extra calculations after the primitives (xo,yo), 9, and A/ are obtained. 
The only memory needed for the recovery is that of the recovered image itself in which the 
edge elements are represented with their associate extra information 9 and A/ . 

Our methodology extends the conditions for detecting stable edge curves from 
bandpassed data set by Torre and Poggio.® They added to the zero detection from S{x,y) 
the requirement that |5(x,?/) | ^ 0 for the detected zero elements. Thus, a detector designed 
to extract zero crossings as edge elements from bandpassed images should also include 
information about the gradient of the bandpassed data near the detected zero crossings 
(usually referred to as slope). This additional information enables the detection of smooth 
and connected edge curves, defines corners in the image, and thresholds the noisy and 
disconnected elements from the true edge elements. In our low-level processing we changed 
the slope evaluation of the Mach-band pattern to its amplitude measurement. This slight 
change made the difference between the recovery of only the primal sketch description to the 
recovery of the intensities between these boundaries as well. 
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E. Accuracy and Stability 

Inaccuracies in extracting the edge primitives introduced into the recovery process, 
even for an ideal case, are caused by the digital implementation of the (mathematically) 
continuous process. Further errors are introduced, in practice, by the image-gathering 
degradations (aliasing, blurring, and noise). It is necessary to choose the interval of 
processing (discretization interval) to be sufficiently small relative to both the sampling 
interval and cr, in order to minimize the inaccuracies of measuring S{x,y,a) and to assure 
stable estimates. 

A recovery process that is based on estimates obtained from Eq. (7) or from Eq. (8) 
seems, initially at least, to be the same. However, estimating A/ from Eq. (8) is like 
estimating it from S'{x,y) (the gradient of S{x,y)), as opposed to the amplitude of S{x,y), 
cis we do in Eq. (7). The former estimation tends to be unstable, as was also observed by 
Hummel,^ especially when the estimate is made for r <C cr. The instability occurs because 
S'{x, y) = reaches its maximum at the edge location {xq-, yo) which is near to where 

we measure S{x, y) to approximate A/ from Eq. (8) [Fig. 4(a)]. By contrast, S'{x, y) is zero 
at (xo ± a sin 0, j/o i <t cos 0), where we measure S{x, y) for the approximation obtained from 
Eq. (7). Small inaccuracies in measuring S near the edge location, where the gradient is the 
steepest, are amplified as a function of A/ and result in large local errors when estimating 
A/ (unstable recovery). On the other hand, small and local inaccuracies in measuring S 
near r = a are insignificant in the overall recovery (stable recovery). 

The relative error in estimating A/ from the Mach-band pattern is 


^A7 

A/ 



( 9 ) 


where b = cr/r, and r denotes the distance from the edge location to the location where 
S was measured. Fig. 4(b) illustrates the relative error e^jjAI as a function of 1 — 1/6 
and a. As expected, the relative error is zero for 1 — 1/6 = 0 (i.e., at r = a). Stable 
estimates can be obtained if the peak and trough of the Mach-band pattern are at a distance 
of at least three intervals of processing from the edge location (i.e., 3 Ax = a and Ax 
denotes the interval of processing. Ax < 1). Hence, the relative error is stable up to 
1 — 1/6 = Ax/2<r, Ax/2(7 = 1/6, (i.e., a — r < Ax/2 and 3Ax = a), and diverges when 
1 — 1/b > Ax 12a. A value of Ax /2a = 1/6 results in an interval of processing that assures 
smooth discrete representation of the continuous For such an interval of processing, 

the local relative error in estimating AI is smaller than 30% [see Figs. 4(c) and 4(d) for an 
actual a], and can be controlled by averaging all the estimates for a given region. 

Therefore, for a stable recovery, we reconunend (1) implementation of filtering and 
processing with a discretization interval Ax < 1 that also obeys Ax/2<t < 1/6, a > ag 
and (2) the use of the estimates of AI obtained by Eq. (7). Eq. (8) may be used only when 
the actual peak or trough location falls between two discrete intervals of processing. After 
the entire image is recovered and a region remains with no estimate at all for its (relative) 
intensity, only then is it recommended to use the estimates of Eq. (8) to complete the 
recovery process (optional second stage of recovery). That way, the (relative) intensity of 
some small spatial details might be inaccurate, but the error will be local with no further 
propagation. 
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3. PRIMITIVE EXTRACTION FROM THE INTENSITY-DEPENDENT 

SPATIAL SUMMATION FILTER 

A. The IDS Filter 

Adaptive response to the intrinsic noisiness of light (photon noise) in early human vision 
processing motivated Cornsweet and Yellott^ to develop the IDS filter. The IDS model 
consists of nonnegative, spatially homogeneous, circularly symmetric spread functions (SF) 
K, with unity volumes. The SF’s differ from each other by their spreads, which are inversely 
dependent on the local intensity I{x,y) as given by 


r(x,y;/) = /A(/r2) 


( 10 ) 


where Thus, the effect of the input intensity is to rescale the SF’s, leaving 

their basic form unchanged [Fig. 5(a)]. The image response to the IDS model is the sum of 
the SF’s [Fig. 5(b)]: 


— oo 

S{x,y;I) = jj T{x',y'-,I)dx'dy'. (11) 

OO 

The IDS operator is an isotropic spatially variant bandpass filter that exhibits the 
following properties: 

(1) Its response to a nonzero uniform scene is unity 

(2) Its response is invariant under translation and rotation 

(3) Its response to an input intensity c/(x, y)c > 0 is 


S[x,y■cI{x,y)] = S[x^/c, yy/c;I {xfy/c, y/Vc)] 


That is, the height of the SF is increased by the factor c while its width is decreased by the 
factor I/a/c (scaling property). 

In the discrete digital implementation, c is chosen so that the diameter of the SF for the 
highest intensity of the image overlaps with at least seven discrete image data. The distance 
between these data is then assumed to be l/y/c. It can be interpreted as the physical 
separation between the sampled data (i.e., the sampling interval), or as the discretization of 
the continuous IDS model given by Eq. (11) (i.e., the interval of processing). 


B. Step Edge Response 

For the recovery purpose, we restrict ourselves to the family of feasible SF’s for the IDS 
filter that are also separable. Similar to the response of an ideal step edge to the y^Gfilter, 
the edge response to the IDS filter, as given by 
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a/Z+A/x ^/Tx 

S{x-I,AI) = l+ J K{z)dz - J K{z)dz = S (yn-,l,W^ , (12) 

0 0 

is a Mach-band pattern around the edge boundary. W = AI j I denotes the Weber fraction 
or the reflectance ratio. The Mach-band pattern crosses the value of one exactly at the edge 
location. The corresponding peak and trough are located symmetrically at a distance p that 
satisfies the equation 


VTTWK (vi + Wp \ = K(p) 


(13) 


where p' = \/Tp, and they take on the value S'(p^; 1, W). The range of the amplitude of the 
Mach-band pattern is dictated by the unity volume condition of the SF’s and is given by 
1 — 0.5 < S{y/Tp; 1, W) <1-1- 0.5. The amplitude reaches its limits 1 > 0.5 when I —y oo. 
The IDS response has a constant amplitude for a constant reflectance ratio W, while its 
spread is also a function of the original intensities I and / -|- AI (Fig. 6). 

The IDS-bandpassed data S{x,y,I,AI) retains information about the original image. 
Thus, the original intensities on both sides of the edge and the reflectance ratio associated 
with each edge element may become a part of the image primitives in the low-level processing. 
The reflectance ratio is recovered by measuring the amplitude of the Mach-band pattern and 
deriving W from Eq. (12). The original intensities are recovered by measuring the distance 
from the peak and trough locations to the one-crossing location. Substituting these distances 
and the estimation of W in Eq. (13) and solving it, we extract the / and / -f AI primitives 
needed for the recovery. 


C. Recovery From the Cylindrical IDS Response 

The cylindrical function is a feasible SF for the IDS-recovery process. Its definition is 

K{x,y) = {^ 0< + < 1/^ 

1 0 elsewhere 

while its corresponding line spread function is 

K{x) = I 2 (i - ' 

lo 

We have chosen to analyze the IDS recovery 
following reasons: 

(1) The cylindrical SF has a finite support. Therefore, the spread of the corresponding IDS 
operator is finite with a radius of (tt/)”^/^. Finite support assures accurate integration in the 


0 < a; < I/a/^ 
elsewhere 

process with the cylindrical SF for the 
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discrete implementation of Eq. (12). SF’s with an infinite support, such as the Gaussian, 
typically would require more than twice the processing to approach the same numerical 
accuracy, due to the larger support necessary for the integration. Accurate integration is 
mandatory to a stable recovery process, as we will show in subsection E. 

(2) The primitives can be extracted explicity from the cylindrical- IDS Mach-band pattern, 
i.e., through a direct relationship between the Mach-band amplitude and the primitives. 
Primitives can be extracted only implicity from the Gaussian-IDS Mach-band pattern, i.e., 
through look-up tables.^ 

Substituting the cylindrical SF in Eq. (13), and solving it for p, we have the distance 
from the one crossing (edge location) to the peak location in the Mach-band pattern as 


p = [7t/(2 + w)] 

The corresponding peak value derived from Eq. (12) takes on the value of 


(14a) 


S(p) = ! + -».„ 


7T 


sm 

7T 


1 


\ 1/2 
W + 2J 


(14b) 


Consequently, the low-level processing consists of estimating the local edge direction 9 at 
the one-crossing location (xo,yo), a^nd measuring the peak and trough values S{p) at their 
corresponding locations (xodbp sin 6,yo:tp cos 9). We can then recover the original ideal edge 
parameters ^dge primitives) W = AI fl, I, and A/, using the bandpass signal information 
(i.e., p and S{p)) from the relationships 


W = = [2p2(VF -h 2)]-\ AI=WI 

1 — sin <p 


(15) 


where (j> = 7r[5(p) — 1] for the peak measurements and (j) = 7r[l — S{p)] for the trough 
measurements. The primitives [xo,yo), kF, and I are used later to recover the (relative) 
reflectance and the intensity representations of the original image. 


D. The Recovery Process 

The recovery process from the IDS-bandpassed data is similar to the recovery process 
described for the V^G-bandpassed data. The initial low-level processing associated each edge 
point (xo, yo) with its local edge direction 9, local reflectance ratio VF, and local intensities 
I and I -f A/. Spreading these primitives onto their corresponding region in a similar 
process described in Section 2.D would result in the original intensity representation and the 
(relative) reflectance representation. The latter representation can be correct only up to a 
constant factor that relates to the initial reflectance that started the spread processing. 
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E. Resolution Accuracy and Stability 


Resolution 


The discrete display elements of the input data I(xi,yi) determine the discretization 
intervals of the discrete IDS-bandpassed output image S{xi,yj). The scaling property of the 
IDS helps us understand the transformation between the continuous representation of the 
IDS model and the discrete representation of the image data. We choose the scaling c so the 
diameter of the smallest spread of the IDS operator (e.g., at the highest intensity) will overlap 
at least seven discrete image elements. Thus, the distance between two discrete elements Ax 
(smaller or equal to the sampling interval) has a physical distance of Ax — \ j y/c. Therefore, 
when we measure the distance p in terms of intervals of processing, we should multiply it by 
1/y^ to be able to use Eq. (15) that was derived from the continuous representation of the 
IDs model. We do the same when we assess the resolution of the image recovery process. 

The distance p^j. between the edge and peak locations in terms of intervals of processing 

is [tt Ax 1(2 + For images that are sufficiently sampled, features larger than 

can be reliably recovered with the primal sketch description, along with the recovery of the 
(relative) reflectance or intensity representations. Spatial features smaller than are 

blurred by the filter. Fig. 7 illustrates the resolution of the recovery for each intensity as a 
function of the scaling parameter c for a given reflectance ratio W . 


Accuracy 


The accuracy with which 
accuracy with which the peak 


the reflectance ratio W can be recovered 
and trough values of Si^x^y) are measured; 


depends on the 


tw 


27T cos <j) 

(1 — sin ^)^ * 


= Tz{W + 2)^/WTlts, 


where <j> = 7r[S'(p) — 1]. The peak and trough curves of the Mach-band pattern are parallel 
to the edge curve. Therefore, a small error in estimating the local edge direction 0, which 
determines the search direction for the peak and trough, does not affect the value S{p). 
Moreover, if for an accurate the actual peak or trough falls between two intervals of 
processing, then one of the neighboring elements is often a better measurement that can 
be obtained easily. Therefore, the error depends mostly on the quantization used in the 
digital implementation of the IDS (8 bits in our case) and is less than 30% for 0.2 <W < 128 
(on the [0,255] range), assuring a stable recovery [Fig. 8(a)]. 

The recovery of the original intensity, on the other hand, depends strongly on the accuracy 
with which the distance p is measured. The dual relationship between I and p as given by 
'kIjP‘{2 + W) = 1, leads to the relative error c/// in estimating I. For a measured distance 
p and an estimated W through a measurement of 5(p) the relative error is 


£7 

I 



(1 + W)tl -t- 47 t/(W + 2) 
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where Cp < l/4c [Figs. 8(b) to 8(d)], The second term of c//7 is the dominant one. It can 
become high for orientations 6 that do not agree with the rectangular lattice or parameters c 
that are not high enough. On the other hand, choosing a higher c would further blur the fine 
details so they would be unrecoverable. The scaling parameter c, which sets the limitation 
on the resolution of the recovery clearly demonstrates the usual trade-off between higher 
resolution (smaller c) and the accuracy of the recovery that can be obtained by increasing c 
[Figs. 7 and 8]. Accurate intensity recovery depends strongly on the type of the original scene 
and on the digital implementation of the IDS operator. Initial inaccuracy in the integration 
will further degrade the estimates of / due to errors in determining the one crossings and 
the distortion of the symmetry of the Mach-band pattern around it. 


Stability 


For each edge element that we recover from the IDS-bandpassed data we have two 
estimates for both the reflectance ratio W and the original intensity I [Eq. (15)]. One 
estimate is from measuring p and S{p) at the peak of the Mach-band pattern, and the other 
one is from measuring p and S{p) at the trough of the Mach-band pattern. We consider an 
estimate of IF or / to be stable when the estimation through the peak and trough are about 
the same. The estimation that we use for the recovery process is the average of the two. 
Unstable estimates may occur when: 

(1) The feature that we try to recover is smaller than 2p^j. 

(2) Two Mach-band patterns interfere (corners, crossings, etc.) 

(3) Edge orientation causes large inaccuracies in measuring p 

(4) The reflectance ratio W is too high 

For a stable recovery we go through a process that is similar to the y^f^recovery. We 
begin recovering the original image using only the stable estimates obtained from the IDS- 
bandpassed data. After the entire image is recovered and a region remains with no estimate 
at all for its reflectance or intensity (depends on which representation we wish to recover), 
only then do we use the unstable estimates to complete the recovery process. For images 
that are sufficiently sampled, regions with no stable estimate are minimal. While all we need 
is one stable estimate to recover a feature without using unstable estimates, we actually have 
stable estimates (that are averaged for smooth appearance) from most of the edge elements. 
The intensity recovery, although stable for images that are sufficiently sampled, tends to 
be less accurate than the (relative) reflectance recovery due to the inevitable discretization 
process. 


4. RESULTS 

In this section we characterize the images that are recovered from both the y^G and IDS- 
bandpassed signals without the use of any other data. The original image data are either 
computer-generated or obtained from a mock setup in space. Together these targets present 
a variety of different scene characteristics. These results combine the accuracy and stability 
assessments for a full recovery and the trade-offs for cases when only partial recovery can be 
obtained. 
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Figure 9 summarizes the recovery of image characteristics from the bandpassed data. The 
original targets are computer generated targets with features that are coarse enough relative 
to the sampling interval. The bull’s-eye target that simulates staircase edges allows us to 
examine how constant change in the edge direction affects the quality of the recovery. The 
square target which is tilted 30 ° relative to the lattice allow us to examine how this tilt affects 
the recovery. The square and the random rectangles targets also allow us to examine how 
corners and crossings in a full range intensity image affect the recovery process. The spread 
of the filter used herein was controlled by <t = 1.0 and the spread of the cylindrical 

IDS was controlled by c = 6000. 

Using only the bandpassed data, the first stage in the recovery process is to extract the 
location of the intensity transitions (edges) through the zero crossing of the V^G-bandpassed 
data or the one crossings of the iDS-bandpassed data. The resulting primal sketches are 
illustrated in Fig. 9(b). The recovery of the (relative) intensity representation from the 
V^G-bandpassed data is illustrated in Fig. 9(c). The quality of the recovery is measured by 
the cross correlation p between the original target and the recovered one. The recovery of the 
intensities and the (relative) reflectance representations from the IDS-bandpassed data are 
illustrated in Figs. 9(d) and 9(e), respectively. As expected from the accuracy assessment, 
the quality of the (relative) reflectance recovery is better than the IDS intensity recovery. 
The high correlations between the original targets and the recovered ones suggest that the 
recovery process may be used as a decoder for a coding scheme in which only the edge 
primitives are transmitted. 

Figure 10 illustrates a particularly important characteristic of the IDS filter, namely, the 
robustness of its reflectance representation to local variation in illumination (e.g., shadow). 
The recovered target [Fig. 10(d)] resembles the original [Fig. 10(a)], and not the shadowy 
[Fig. 10(b)], which is the one that was filtered. Traces of the shadow degradation can be 
seen in the modest loss of accuracy in the actual transition as the illumination decreases. 

Figure 11 illustrates the capability of the recovery process with an experimental setup 
that simulated imaging conditions in space. The target examines the recoverability of targets 
with a wide dynamic range of intensities, in particular, the recoverability of spatial details 
under direct illumination or in deep shadow. As can be observed from the edge recovery 
representation [Fig. 11(b)], many important features of target, including features in deep 
shadow, could be recovered. The recoveries from the (with a = 3.0) are illustrated in 
Fig. 11(c), and those from the IDS (with c = 6000) are illustrated in Fig. 11(d) and Fig. 11(e). 
These recoveries resemble a slightly blurred version of the original target [Fig. 11(a)]. As 
could be expected from the accuracy and stability analysis, features that were relatively small 
produced inaccuracies as well as features with abrupt and steep changes in the intensities. 
Nevertheless, in most of the cases, these inaccuracies were contained within their regions 
with no further propagation. 


5. CONCLUDING REMARKS 

Assessment of the response of an ideal edge to a bandpass filter reveals that most of 
the target’s characteristics are preserved. For applications that use bandpass signals, the 
recovery of those characteristics exhibits new dimensions to image understanding. Minimal 
extra processing is required, beyond that is needed for the primal sketch extraction process. 
The processing is based on the existing bandpassed data information and on the same 
storage needed for the recovered image itself. The potential for high data compression 
applications, transmitting only the information associated with detectable edge boundaries 
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(edge primitives), might be helpful when high data rate transmission is required. In light 
of our results and the stability assessment, we feel that the edge primitives information 
extracted from the bandpassed image is a good form of representation for images that are 
sufficiently sampled and properly processed. 
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Figure 1: The recovery process from the bandpassed representation to the original image 
representation. 


(a) Spatial domain (b) Frequency domain 

Figure 2: Normalized response for three standard deviations a. 
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Figure 3: Normalized response to an ideal step edge for \/‘^G operators with different a 
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Figure 4: Accuracy of the recovery process. 
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Figure 5: The IDS response. 
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Figure 6: Response of ideal step edges to the IDS operator. 
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Figure 7: Resolution of the IDS recovery process. 
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Figure 8: Accuracy of the IDS recovery process. 
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Figure 9; The recovery process for computer-generated targets 
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Figure 10: Recovery from spatially varying illumination 
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Figure 11: Feature extraction for an experimental image that simulates imaging conditions 
in space. 
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